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© Controlled-feedback packet switching system. 

@ Packets or cells received from different input 
ports (101,, 101 J of a switch (102) and destined 
for a common output port (103! - 103 n ) of that 
switch, are analyzed to determine their priority 
level. Lower-priority packets or cells are buf- 
fered in recirculation delay lines (105, - 105J of 
appropriately-selected lengths, and thereafter 
scheduled for transmission to the output port 
based on their level of priority. 
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Technical Field 

This invention relates to switching systems. More 
specifically, this invention relates to the scheduling of 
packet delivery in a switching system. 

Background 

Because of the unscheduled nature of arrivals of 
packets or ATM cells to a packet switching system, 
two or more packets may simultaneously arrive on 
different inputs destined for the same output. The 
switch architecture may allow one of these packets to 
pass through to the output, but the others must be 
queued for later transmissions. This temporary con- 
gestion caused by simultaneous arrival of packets or 
cells is typically handled by temporarily storing the 
packets or cells in buffers. For traffic distributions 
that are random or more or less uniform, buffering re- 
quirements are rather lenient. However, for high per- 
formance-packet switching systems designed to han- 
dle bursty traffic, the buffering requirements are 
more stringent 

For electronic packet switches, buffering is ordi- 
narily implemented in a random access memory 
(RAM) that is typically shared by all the inputs and 
outputs of the switch in order to reduce memory stor- 
age requirements, in optical packet switches, the 
present lackof an optical random access memory sig- 
nificantly complicates buffering in those optical 
switches. Approaches that have been considered for 
buffering in optical switches include an implementa- 
tion that involves the routing of queued packets to 
trap lines that retard the transmission of the queued 
input packets to the desired output, thereby allowing 
other input packets destined for the same output to be 
transmitted during the delay period. However, this ap- 
proach presents certain drawbacks that prevent its 
use in optical and optoelectronic packet switches. 
Specifically, certain scheduling functions needed for 
the orderly and timely switching and transmission of 
packets are not performed in the trap line approach. 
Fo r examp l e, the t r ap li ne approach (u nli ke the RAM 
approach) does not permit changes to the "sched- 
uled" transmission time of lower- priority packets 
when higher-priority packets arrive later. 

Another approach that has been advocated for 
buffering packets in optical switches is the so-called 
"feed-forward" technique in which packets contend- 
ing for an output port are delayed by different num- 
bers of time slots to avoid collision with previously 
scheduled packets. In that approach, packets are 
dropped if they cannot be scheduled in a collision- 
free manner. This approach, however, does not allow 
transmission time to be updated on a slot-by-slot ba- 
sis and does not adequately support priority traffic. 

Thus, there is a need for a packet buffering sys- 
tem for use in optical and optoelectronic packet 



switches which offers the same performance and 
functionality provided by a RAM in electronic packet 
switches. 

5 Summary 

This invention is directed to a packet switch in 
which incoming packets or cells destined for a com- 
mon output port are analyzed to determine their pri- 
10 ority level. Lower-priority packets or cejls are buf- 
fered in recirculation delay lines of appropriately-se- 
lected lengths, and thereafter are scheduled for 
transmission to the output ports based on their level 
of priority. 

15 In a specific example of the invention, a memory- 

less non-blocking switch is designed to include an in- 
put/output section comprised of a certain number of 
input ports and output ports and a certain number of 
recirculation delay lines of various lengths connected 

20 to dedicated input and output ports for buffering pack- 
ets. The switch also includes control circuitry that der 
termines which packets need to be buffered in the re- 
circulation delay lines and schedules the delivery of 
the buffered packets based on their priority level. The 

25 control circuitry also keeps packets in their proper 
first- in, first-out sequence, supports multiple levels of 
priority traffic, and ensures that packets pass 
through the recirculation delay lines only a small num- 
ber of times to minimize power losses, thereby avoid- 

30 ing the need for optical amplifiers in the delay lines in 
most cases. If amplifiers are used, a reduced number 
of lines through which a packet has to travel results 
in a proportional decrease in amplifier noise. 

In another example of the invention, a packet 

35 switching system is partitioned into multiple, memory- 
less, non-blocking switches that are either connected 
to recirculation delay lines or to the output ports of the 
packet switching system. The first group of switches 
are called a "delay-line switches" while the second 

40 are called an "output switches". Delay-line and output 
switches are front-ended by routers which receive 
commands from a control circuitry to direct packets 
f r om input ports to cither q de l ay lino cw i tch wh e n the 



packets must be queued or to an output switch when 
45 no queuing is required. 

Brief Description of the Drawing 

FIG. 1 illustrates a packet switch designed in ac- 
50 cordance with the invention to schedule delivery 

of lower-priority packets based on their priority 
tevei and to buffer packets in recirculation delay 
lines of appropriately-selected lengths; 
FIG. 2 shows one example of a control table 
55 which keeps track of packets in the recirculation 

delay lines of FIGs. 1 and 3; and 
FIG. 3 shows an illustrative packet switching sys- 
tem that is partitioned into multiple switches 



2 



EP 0 634 854 A2 



front-ended by routers. 
Detailed Description 

FIG. 1 illustrates an example of a packet switch 5 
which schedules delivery of packets based on their 
priority level. The packet switch of FIG. 1 also buffers 
lower- priority packets in recirculation delay lines of 
appropriately-selected lengths. FIG. 1 shows a block 
diagram of a nxn packet switch comprising a) n Input 10 
lines 101-1 to 101-n b) n output ports 103-1 to 103-n 
c) m recirculation delay lines (of appropriately-select- 
ed lengths) 105-1 to 1 05-m for buffering packets d) an 
(n+m)x(n+m) memoryless, non-blocking switching 
block 102, and e) a control circuit 104 which reconfi- 15 
gures the switching block 102 on a packet-by-packet 
basis. In FIG. 1, packets arrive at the input lines 101- 
1 to 101-n of the switching block 102. While switching 
block 102 can be a strictly non-blocking switching 
block, it is preferably a rearrangeably non-blocking 20 
switching block. For optical implementations, input 
lines 101-1 to 101-n may be fiber lines. When there 
is no contention for the output ports, packets received 
from input lines 101-1 to 101-n are transmitted to the 
appropriate output ports 102-1 to 102-n based on the 25 
packet headers that indicate the destination point for 
each packet The header of a packet is read by control 
104 which uses power splitters 110-1 to 110-n to tap 
a small fraction of the packet energy to read the head- 
er. To determine the appropriate configuration of the 30 
switching block 102, control 104 keeps track of all 
packets buffered in the recirculation delay lines 105- 
1 to 105-m, so that it knows which packets will be re- 
turning to the switching block 102 at whattimes. Con- 
trol 104 may be implemented using, for example, a 35 
microprocessor which executes programming in- 
structions described below. 

FIG. 1 shows that the recirculation delay lines 
105-1 to 105-m have lengths d 1( d 2 , and d m . The 
lengths of those lines are expressed in units equal to 40 
the number of packets that they respectively can 
store end-to-end. In order to explain the interaction 
between switching block 102, control 104 and recir- 
culation lines 105- 1 to 1Q5-m, it is assumed in this ex- 



in the recirculation delay lines 105-1 to 105-m. A pack- 
et that is buffered in a recirculation delay line of length 
d| will exit that delay line and return to switching block 
102 after d, time slots. Control 104 selects which 
packets to output for each time slot, and also assigns 
the remaining packets to the recirculation delay lines. 
More specifically, control 104 assigns packets to the 
appropriate delay lines in a way such that "buffer lo- 
cations" in the delay lines are efficiently utilized, and 
packets retain thetr proper first- in, first-out sequence, 
as needed. Control 104 also ensures that each pack- 
et circulates through the delay lines 105-1 to 105-m 
only a small number of times. This last property is 
very important, because it may help to keep the pow- 
er losses low enough to reduce, and perhaps elimin- 
ate, the need for optical amplifiers in the recirculation 
delay lines 105-1 to 105-m. If amplifiers are needed 
in the recirculation delay lines 105-1 to 105-m, mini- 
mization of the number of recirculations helps keep 
the added noise sufficiently small. In addition, if the 
power budget does indicate the need for amplifiers in 
the recirculation delay lines 105-1 to 105-m, the re- 
duction in the number of lines translates into a valua- 
ble reduction in the number of amplifiers. 

It may be important to note that many different 
technologies can potentially be used for an optical im- 
plementation of the switch illustrated in FIG. 1. For ex- 
ample, technologies such as guided- wave using li- 
thium niobate; or perhaps Wavelength Diversion Mul- 
tiplexing (WDM) and star couplers; or even wave- 
length routers can be used to build such a switch. In 
addition, the delay-line function that is performed in 
this example by the recirculation delay lines 105-1 to 
105-m can be implemented using, for example, opti- 
cal fiber lines or other appropriate optical transmis- 
sion means. As to the control circuitry 104, it can 
more easily be implemented using a microprocessor 
even though it is also possible to use optical logic 
gates to design such a circuit in an optical implemen- 
tation. 

FIG. 2 shows one exemplary structure of a con- 
trol table which is arranged to keep track of packets 
in the recirculation delay lines of FIG. 1 described 
above and FIG. 3 described below. 



ample, (without loss of generality, and for the sake of 
simplicity) that d 1 ^ d 2 = — = d m . For illustrative pur- 
poses, special attention is devoted to the case in 
which d, = 1, d 2 = 2 and d m = m. Since a total of 

B 4 (di + d 2 + — + dm) packets can be stored in the 
recirculation delay lines 105-1 to 1 05-m, it follows that 

B = m(m+1)/2 when d n = 1, d 2 = 2 and d m = m. It 

is further assumed that switching block 102 and con- 
trol circuit 1 04 use a time-slotted system in which, for 
each time slot, up to (n+m) packets may arrive at the 
switch (i.e., n new arrivals plus m "recirculation pack- 
ets"). Of these packets, up to n can be transmitted to 
their appropriate output ports; the rest are "buffered" 
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In order to free control circuit 104 of FIG. 1 of the 
task of reading (for each time slot) the headers of all 
packets exiting the recirculation delay lines 105-1 to 
1 05-m, control circuitry 104 maintains a table of infor- 
mation about the packets stored in the recirculation 
delay lines 105-1 to 105-m. FIG. 2 illustrates one pos- 
sible way for control 104 to store information about 
the buffered packets. As mentioned above, this infor- 
mation (e.g., the packets' output port addresses) is 
obtained by reading the packet headers when they 
first arrive to the switching block 102. This information 
is kept in the control table of FIG. 2 until the packet 
exits the switching block 102. In this example, the ta- 
ble of FIG. 2 is an mxd m table in an (electronic) RAM, 
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that operates at the packet rate and that is arranged 
to mimic the flow of packets through the recirculation 
delay lines 105-1 to 105-m. Each row in the table cor- 
responds to one of the m delay lines (m = 8 In FIG. 2). 
Each column contains information corresponding to s 
the set of packets that will exit the delay lines at the 
same time and arrive together for routing through the 
switching block 102. The time-slot pointer shown at 
the bottom of FIG. 2 cyclically shifts to the left one 
column per time slot to indicate the current set of (up 10 
to) m recirculation packets now returning to switching 
block 102. The shaded portion of the mxd ra table cor- 
responds to unused entries (for a given position of the 
time-slot pointer). When a packet is buffered in a re- 
circulation delay line, information about that packet is 15 
moved in the control table of FIG. 2 to the appropriate 
position along the "diagonal" (P u for delay line 1 , 

for delay line 2, P 3e for delay line 3 or P 83 for delay 

line 8). The amount of information associated with 
each of these P,j entries depends on the particular im- 20 
piementation of control 104, as illustrated by the fol- 
lowing two examples. 

In an illustrative non-FIFO (First-ln First-Out) 
control implementation, the control 104 first routes as 
many of the packets to their outputs as possible for 25 
each time slot. In sequence, it considers the recircu- 
lation packets (beginning at the longest delay line and 
moving towards the shortest delay line), and then 
considers the new packet arrivals. If there are multiple 
priorities, higher-priority packets are handled first 30 
Whenever a packet cannot be routed to its output port 
j (because another packet has already been selected 
for routing to output j), then the packet is sent to the 
shortest delay line that has the fewest packets des- 
tined for that output j "in that column" of the Control 35 
Table of FIG. 2. Since at most one packet per output 
can be transmitted for each time slot, this last factor 
helps "load balance" the output addresses over the ta- 
ble's columns. 

In an illustrative FIFO-control implementation, 40 
for each time slot, the new packet arrivals and the re- 
circulation packets are either (i) routed to the appro- 
priate output ports for transmission, (ii) "scheduled" 



for transmission after one more recirculation, or (iii) 
left "unscheduled" and sent to a delay line for another 45 
recirculation. Each P,j entry in the control table keeps 
track of (i) the input-output ports of each packet in the 
recirculation delay lines, and (ii) whether or not the 
packet is scheduled for transmission the next time it 
reaches the switching block 1 02. There also is a FiFO 50 
table associated with each input-output pair. The 
FIFO keeps a f irst-in, first-out list of all packets of this 
input-output pair, the exact location of these packets 
in the recirculation delay lines and in which (if any) fu- 
ture time slots they are scheduled for transmission. 55 
Finally, a "timestamp" is given to each packet when it 
first arrives to the switching block 1 02. The timestamp 
corresponds to the transmission time of an ideal out- 



put-queuing switch, and represents the packet's "an- 
ticipated transmit time." 

Using this information about each packet, the 
Control schedules packets for transmission in the 
current time slot or future time slots. First, it routes 
any "scheduled packets" on the recirculation delay 
lines to the appropriate output ports. Second, it 
checks if any of the new arrivals can be routed to their 
outputs (without violating the FIFO constraint). Third, 
it schedules packets, If possible, for transmission af- 
ter their next recirculation (on the shortest possible 
delay line). A packet can be scheduled only if the 
packet before it in its input-output FIFO has already 
been scheduled. Priority in this scheduling is given to 
packets with the smallest timestamp ("anticipated 
transmit time"), and to recirculation packets on the 
longest delay lines. Once a packet is scheduled, this 
may also indirectly allow other packets waiting in the 
same input-output FIFO to be scheduled without vio- 
lating the FIFO constraint Finally, any remaining 
packets are left unscheduled and are recirculated to 
the delay lines. Starting with unscheduled packets 
having the smallest timestamp, packets are sent to 
the shortest delay line that has the fewest packets 
destined for that output in that column of the control 
table of FIG. 2. 

Other contention resolution schemes considered 
include routing a packet to an output port (as opposed 
to a recirculation delay line) based on the position of 
the input port from which the packet is received when 
that packet is contending for an output port with an- 
other packet of equal priority level. Likewise, a packet 
received from a recirculation delay line dedicated in- 
put port may be routed to an output port when another 
packet received simulatneously from a "regular" input 
port is contending for the same output port In that 
case, in accordance with the principles of the inven- 
tion, the other packet would be routed to a recircula- 
tion delay line, priority 

FIG. 3 shows an illustrative packet switching sys- 
tem that is partitioned into multiple switches front- 
ended by routers. The modular design, illustrated in 
F I G . 3, p a rtitions a paokot switch i n t o a "memory — 
block" comprised of memoryless non-blocking 
switching blocks (MNSB) 306, 312, 314, and 330, de- 
lay lines 318-1 to 318-m. and an input/output section 
comprised of input lines 301-1 to 301-n and output 
ports 340-1 to 340-n, respectively. In FIG. 3, the 
(n+m)x(n+m) switch of FIG. 1 is configured as'nxn, 
nxm, mxn, and mxm MNSBs, plus a number of rout- 
ers 303, 304, 319, 320 and optional rear-end switches 
309, 310, 316 and 317 for certain implementations. 

The packet switching system that is illustrated in 
the block diagram of FIG. 3 includes routers 303, 304, 
320 and 31 9 that directs packets to either a delay-line 
MNSB, such as MNSB 314 (330) or an output MNSB 
such as MNSB 312 (306). Each MNSB is connected 
to n routers that receive incoming packets from input 
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lines 301-1 to 303-n or m routers that receive recircu- 
lated packets from recirculation delay lines 318-1 to 
318-m. In an optical implementation of the packet 
switching system of FIG. 3, routers 303 and 304 also 
perform a power splitting function that allows a con- 5 
trol 307 to tap a small fraction of a packet's energy to 
read that packet's header. The packet switching sys- 
tem that is illustrated in the block diagram of FIG. 3 
shares some common elements with the packet 
switch shown in FIG. 1. For example, the packet 10 
switching system of FIG. 3 uses a) the same set of re- 
circulation delay lines shown in FIG. 1, and b) a sim- 
ilar control structure, albeit with a slightly more dis- 
tributed implementation. Thus, the control table illu- 
strated in FIG. 2 is readily applicable to the modular 15 
design of FIG. 3. 

As is the case for the switch of FIG. 1 1 the Control 
307 is responsible for all routing decisions, including 
deciding which delay lines buffer which packets. For 
example, when multiple incoming packets destined 20 
for the same output port are received by routers 303 
or 304 from input lines 301-1 to 301-n, the headers of 
these packets are read by control 307 which decides 
the proper treatment for the received packets using, 
for example, the scheduling techniques described 25 
above. As part of that decision, control 307 determi- 
nes which packet among the received packets to 
send to the appropriate output port based on the pri- 
ority level of the packets. The selected packet to be 
routed to the output port is sent to MNSB 306 which 30 
promptly forwards that packet to rear-end switch 309 
or 310 via line 308. Rear-end switch is used in optical 
implementations of the packet switching system of 
FIG. 3 to prevent energy loss that could result from 
the use of other types of devices. When a packet has 35 
to be buffered, control 307 sends a signal to router 
303 or 304 to forward that packet to MNSB 330 which 
in turn sends the packet to the delay line indicated by 
control 307. 

Advantageously, buffering packets in the recircu- 40 
lation delay lines (105-1 to 105-m in FIG. 1 or 318-1 
to 318-m in FIG. 3) offers flexibility and efficiency not 
possible with other techniques. For example, if first- 
in first-out requirements dictate that an arriving pack- 



each time a packet returns to the switch from the 
feedback delay lines (e.g., to allow for quicker trans- 
mission of new higher- priority packets). 



Claims 

1. A packet switching system comprising: 

at least one switching block having a plur- 
ality of input and output ports; 

a control circuit for determining a) priority 
levels of packets received by the input ports and 
b) delivery time of packets to the output ports; 

at least one recirculation delay element; 

and 

means for buffering packets in the at least 
one recirculation delay element in accordance 
with the priority levels determined by the control 
circuit. 

2. The invention of claim 1 wherein said buffering 
means comprise 

means for routing at least one lower-prior- 
ity packet to the at least one recirculation delay 
element until the lower-priority packet is no lon- 
ger in contention for the output port with at ieast 
one higher-priority packet; and 

means for keeping track of all packets 
routed to all recirculation delay elements. 

3. The invention of claim 2 further comprising 

means for limiting the number of times a 
lower-priority packet is routed to a recirculation 
delay element. 

4. The invention of claim 1 further comprising 

means for routing a lower-priority packet 
to the output port after at least one higher-priority 
packet has been routed to said output port 

5. The invention of claim 1 wherein the packet 
switching system has more than one recirculation 
delay elements of different time-delaying means 
and wherein a packet is buffered in a particular 



et cannot be transmitted for at least k time slots (be- 45 
cause other packets are queued for the same output), 
then, it is advantageous to store that packet in a re- 
circulation delay line of length k (if possible) and 
transmit that packet in due time. Even if the packet 
cannot be buffered in a delay line of length k, "optimal so 
performance" may still be possible since there may 
be many combinations of delay-line lengths that sum 
to k. For example, an optimal (i.e., minimal) delay of 
10 time slots can be attained by buffering a packet in 
a delay line of length 1 0, or by buffering it successive- 55 
ly in delay fines of length 7 and 3, or successive delay 
lines of length 6, 1, and 3, etc. Furthermore, recircu- 
lation permits "scheduling decisions" to be-revised 



recirculation delay element based on its priority 
level. 

The invention of claim 5 wherein lower-priority 
packets are buffered in longer-time-delaying- 
means element and higher-priority packets are 
buffered in shorter-time-delaying-means ele- 
ment. 

The invention of claim 1 wherein each recircula- 
tion delay element is a fiber-optic line that is of a 
particular length and that is connected to a dedi- 
cated input and output port of a switching block. 
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9. 



The invention of claim 1 further comprising 

means for keeping ail packets routed to an 
output port of the switching block in a f irst-in first- 
out sequence. 

Apparatus for use in a packet switching system 
comprising: 

means for determining the level of priority 
of incoming packets received from different input 
ports of the packet switching system and des- 
tined for a common output port of the packet 
switching system; 

routing a higher-priority packet to the out- 
put port; 

means for buffering in at least one recircu- 
lation delay line at least one lower- priority packet; 
and 

means for scheduling delivery to the out- 
put port of the at least one lower-priority packet 
buffered in the recirculation delay lines. 

10. The apparatus of claim 9 wherein the scheduling 
means comprises: 

means for routing at least once the at least 
one lower-priority packet to the recirculation de- 
lay lines. 

11. The invention of claim 10 further comprising 

means for limiting the number of times a 
lower-priority packet can be routed to the at least 
one recirculation delay line. 

12. A method of routing packets in a switch compris- 
ing 

determining levels of priority of incoming 
packets received from different input ports of the 
switch and destined for a common output port of 
the switch; 

routing a higher-priority packet to the out- 
put port; 

buffering in at least one recirculation delay 
line at least one lower-priority packet; and 

scheduling delivery to thft nntpnt P »rt r f 

the at least one lower-priority packet buffered in 
the at least one recirculation delay line. 

13. A packet switching system comprising: 

a switching fabric having at least one 
switching block with a plurality of input and output 
ports; 

a control circuit for determining levels of 
priority for packets received from the input ports; 

at least one recirculation delay element 
which is connected to a dedicated input port and 
a dedicated output port of the switching fabric and 
through which, a particular packet exiting the 
switching fabric is returned to the switching fabric 
at least once; and 
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means for queuing at least one packet in 
the at least one recirculation delay element in re- 
sponse to a signal from the control circuit 

14. The invention of claim 13 wherein the signal is 
emitted by the control circuit if at least two pack- 
ets are received simultaneously on different in- 
put ports and are destined for a common output 
port. 

15. The invention of claim 13 wherein the signal from 
the control circuit is a function of the priority lev- 
els of the received packets and programming in- 
structions in said circuit. 

16. The invention of claim 1 5 further comprising 

means for routing at least one lower-prior- 
ity packet to the at least one recirculation delay 
element at least once. 

17. The invention of claim 1 5 further comprising 

means for routing a queued packet to an 
output port when the queued packet is in conten- 
tion for the output port with another packet of 
equal priority level. 

18. The invention of claim 15 further comprising 

means for routing a packet to an output 
port based on the position of the input port from 
which said packet is received when said packet is 
in contention with another packet of equal priority 
level. 
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FIG. 2 
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FIG. 3 
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